Accessing deep web informaiton using a search engine

ABSTRACT

Methods, apparatuses, and articles for receiving a search request associated with a transportation service from a client device, the search request including a plurality of search criteria, are described herein. Additionally, the methods, apparatuses, and articles further return to the client device an answer page having a plurality of answers potentially associated with the transportation service, the plurality of answers identifying a plurality of information locations having information potentially associated with at least a one of the plurality of search criteria, where at least one of the answers includes at least one input field of a query answer page for entry of at least one feature of the transportation service, the query answer page to be dynamically generated by one of the information locations in response to a query.

RELATED APPLICATIONS

The present invention is a continuation application of U.S. patentapplication Ser. No. 13/491,459, filed 7 Jun. 2012, entitled “AccessingDeep Web Information Associated with Transportation Services Using aSearch Engine,” which is a continuation application of U.S. patentapplication Ser. No. 12/768,621, filed 27 Apr. 2010, entitled “AccessingDeep Web Information Associated with Transportation Services Using aSearch Engine,” now issued as U.S. Pat. No. 8,321,529, which is acontinuation of U.S. patent application Ser. No. 11/595,090, filed 8Nov. 2006, entitled “Accessing Deep Web Information Associated withTransportation Services Using a Search Engine,” now issued as U.S. Pat.No. 7,716,303, which is a continuation-in-part application of U.S.patent application Ser. No. 10/397,992, filed 25 Mar. 2003, entitled“Accessing Deep Web Information Using a Search Engine,” now issued asU.S. Pat. No. 7,693,956, which claims priority to U.S. ProvisionalApplication No. 60/366,817, filed 25 Mar. 2002, each of which isincorporated herein by reference in its respective entirety.

FIELD OF THE INVENTION

The present invention relates to the field of data processing. Morespecifically, the present invention is related to accessing deep webpages potentially associated with transportation services using a searchengine.

BACKGROUND OF THE INVENTION

Currently, Internet search engines index the information found on staticweb pages on public websites. Static web pages are web pages that existbefore and after a user accesses them. A search engine saves two piecesof information from each static web page it indexes. The first piece ofinformation is the location of the web page, commonly referred to as theuniform resource locator (URL). The second piece of information is oneor more keywords from the text of the web page. A search engine is then,essentially, a collection of URLs and associated keywords.

Deep web information, however, is not contained on static web pages.Instead, it is produced in response to a specific query by a user andresults in dynamic web pages. Search engines themselves are deepwebsites. A dynamic web page only exists after a user's query anddisappears after the user exits the dynamic web page. As a result, deepweb information or the dynamic web pages that can be generated from thedeep web information are not typically available from search engines.

As a result, a user using a search engine to locate information, forcertain type of information searching, often finds himself/herself notfinding the right information, or having to re-enter the searchcriteria, as query parameters of queries to generate the dynamic webpages having the desired information.

FIGS. 8 a-8 d illustrate one example of this problem. For theillustrated example, a user uses a search service/engine (such asGoggle) to search for “Flights from Seattle to San Jose”. FIG. 8 aillustrates selected portions of an example search answer web pagereturned from the search service/engine. An example search answer webpage includes indices and indexing flight deals on Priceline andOrbitz's websites respectively, and indices and indexing staticpromotional pages on Southwest and Alaska Airline's websitesrespectively.

FIG. 8 b illustrates the example static web page indexed by an index. Itis essentially a form for facilitating a user to search for specificflight deals available on Priceline's website. In other words, becausethe flight deal web page for flights between Seattle and San Jose, ifavailable, is dynamically generated based on flight deal information ina database, the proper query answer web page is not indexed by thesearch service/engine. Consequently it is not available. Only thegeneric static root web page (which could have been retrieved withmerely the search criteria of“flight deals”) is indexed, resulting inthe user essentially having to start the search over again, once he/sheenters Priceline's website.

Similar disadvantages exist, if the Orbitz website is entered via anindex.

FIGS. 8 c-8 d illustrate the static promotional web pages indexed byindices. They are merely promotional web pages, and not the flightschedule or availability web pages that the user is looking for. Again,the reason is that the flight schedule and availability web pages aredynamically generated based on the flight schedule and availabilityinformation in the databases of Southwest and Alaska Airlines.

FIGS. 9 a-9 b illustrate the end results that the user desires. That is,in response to the user's provision to a search engine, of the searchcriteria “flights between Seattle and San Jose”, the user is providedwith a search answer page having indices that index not only therelevant static web pages, but also the relevant dynamic web pages to begenerated, such as would be provided using this invention. Further, onselection of one of such indices, the user can be taken to the indexeddynamic web page, with the search criteria preserved, i.e. the from andto cities of Seattle and San Jose, filled in, without requiring the userto re-fill or re-select the search criteria again.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be described by way of exemplary embodiments,but not limitations, illustrated in the accompanying drawings in whichlike references denote similar elements, and in which:

FIG. 1 illustrates network view of a computing environment suitable forpracticing present invention, in accordance with one embodiment;

FIG. 2 illustrates an overview of the method of the present invention,in accordance with one embodiment;

FIGS. 3 a-3 b illustrate a number of reconciled queries with theirassociated query parameters, of a number of websites, for a number ofexample search criteria, created in accordance with the teachings of thepresent invention;

FIGS. 4 a-4 b illustrate a number of example proxy web pagespecifications, created in accordance with the teachings of the presentinvention;

FIGS. 5 a-5 b illustrate two example indices created using proxy webpage specifications of the present invention;

FIGS. 6 a-6 d illustrate a source view and an end-user view each of twoexample pre-filled forms for two example queries employing the POSTmethod to post its associated query parameters to the correspondingwebsites;

FIG. 7 illustrates an example computer system suitable for use as eithera client or to host a proxy function, a search engine or a web server,to practice the present invention, in accordance with one embodiment;

FIGS. 8 a-8 d illustrate the inadequacies of the prior art infacilitating a user in accessing deep web information using a searchengine, for an example information search;

FIGS. 9 a-9 b illustrate two user desired results of the exampleinformation search of FIG. 8 a-8 d;

FIGS. 10 a-10 d illustrate four example search result pages associatedwith car rental service queries, in accordance with an embodiment of thepresent invention;

FIG. 11 illustrates an exemplary fragment of source code of a searchresult page, the page associated with a car rental service query;

FIGS. 12 a-12 d illustrate four example search result pages associatedwith train reservation service queries, in accordance with an embodimentof the present invention;

FIG. 13 illustrates an exemplary fragment of source code of a searchresult page, the page associated with a train reservation service query;

FIGS. 14 a-14 d illustrate four example search result pages associatedwith flight status information service queries, in accordance with anembodiment of the present invention; and

FIG. 15 illustrates an exemplary fragment of source code of a searchresult page, the page associated with a flight status service query.

DETAILED DESCRIPTION

The present invention includes search engines enhanced to effectivelyindex dynamic web pages, companion proxy servers, and related methodsand apparatuses.

Part of the description will be presented in terms commonly employed bythose skilled in the art to convey the substance of their work to othersskilled in the art, including but are not limited to terms such aswebsites, static and dynamic web pages, queries, query parameters, queryanswer pages, search criteria, search answer web pages and forms. On acomputing device, these logical entities are represented and manipulatedin the form of electrical and/or optical signals.

In the following description, various aspects of the present inventionwill be described. However, it will be apparent to those skilled in theart that the present invention may be practiced with only some or allaspects of the present invention. For purposes of explanation, specificnumbers, materials and configurations are set forth in order to providea thorough understanding of the present invention. However, it will beapparent to one skilled in the art that the present invention may bepracticed without the specific details. In other instances, well-knownfeatures are omitted or simplified in order not to obscure the presentinvention.

Various operations will be described as multiple discrete steps in turn,in a manner that is most helpful in understanding the present invention,however, the order of description should not be construed as to implythat these operations are necessarily order dependent. In particular,these operations need not be performed in the order of presentation.

The use of heading and sections are to facilitate understanding of thepresent invention, and they are not to be limiting on the presentinvention. The phrase “in one embodiment” is used repeatedly. The phrasegenerally does not refer to the same embodiment, however, it may. Theterms “comprising”, “having” and “including” are synonymous, unless thecontext dictates otherwise.

Overview

Referring now to FIG. 1, wherein a network view of a computingenvironment suitable for practicing the present invention, is shown. Asillustrated, computing environment 100 includes a number of websites 108offering services and/or information. In particular, each website 108includes dynamic web page generator 118 to generate various dynamic webpages to facilitate delivery of the services and/or information offered.

Typically, dynamic web page generator 118 generates the various dynamicweb pages using information stored in various databases (not shown) ofwebsites 108, which may be co-resident or remotely disposed from theserver(s) hosting websites 108 (but communicatively coupled). Eachdynamic web page is typically generated in response to a query having aparticular set of associated query parameters. Accordingly, thesedynamic web pages will also be referred to as query answer web pages.

Further, the query answer web pages may be generated using CommonGateway Interface (CGI) scripts, in conjunction with instructions of oneor more programming languages (Perl, C, C++, Basic, Fortran, Pascal,etc.), or a proprietary dynamic web page generator, such as Microsoft'sgenerated ASP pages.

For the illustrated embodiment, communication with websites 108 areconducted using the Hypertext Transmission Protocol (HTTP) overTransmission Control Protocol/Internet Protocol (TCP/IP), and thedynamically generated web pages are specified using the Hypertext MarkupLanguage (HTML). For some queries, the associated query parameters aresubmitted to websites 108 using a GET method, i.e. the associated queryparameters of the queries are submitted in conjunction with thecorresponding URLs, identifying the locations of the correspondingwebsites 108. Whereas, for other queries, the associated queryparameters are submitted to websites 108 using a POST method, i.e. theassociated query parameters of the queries are submitted separately(typically subsequently), after corresponding connections have beenestablished with the websites 108, using their corresponding URLs. Thus,in the second method called the POST method, the URL is disassociatedfrom the input parameters. The associated input parameters with the URLwould make the resulting URL unique. However, without a unique URL, thatis, without the association of the FORM URL with the specific inputparameters, the completed FORM cannot be indexed by a search engine orcannot even be bookmarked by a browser.

In alternate embodiments, the present invention may be practiced withcommunication with websites 108 being practiced using othercommunication protocols, including but are not limited to HTTP Secure(HTTPS), File Transfer Protocol (FTP), Secure FTP (SFTP), Telnet, SecureShell, Secure Copy (SCP), Goopher, and so forth.

Continuing to refer to FIG. 1, computing environment 100 furtherincludes a number of clients 102, and search services 104, coupled toeach other and to websites 108 via network 110. Clients 102, assisted bysearch services 104, selectively consume the services and/or informationoffered by websites 108.

For the embodiment, clients 102 include browsers 112, and use browsers112 to consume the service and/or information offered by websites 108 inthe form of web pages. Clients 102 further include operating systemservices (not shown) in support of browsers 112. Browsers 112, supportedby the operating system services, are equipped to communicate withsearch services 104 and websites 108 in one or more of the earlierdescribed protocols.

In alternate embodiments, browsers 112 and the underlying operatingsystem services may be fully integrated. In other embodiments, somefunctions of browsers 112 and the underlying operating system servicesmay be offered by an intermediate “middle” layer of services instead.

For the embodiment, search services 104 include search engine 114, webpage indices 115 indexing web pages of websites 108 offering servicesand/or information, and index generator 113 to generate indices 115. Aswill be described in more detail below, under the present invention, webpage indices 115 also advantageously index web pages of websites 108that are dynamically generated in response to queries having associatedquery parameters.

Similar to clients 102, search services 104 further include operatingsystem services (not shown) in support of elements 113-115. Searchengine 114 supported by the operating system services, are equipped tocommunicate with clients 102 and websites 108 in the appropriateprotocols.

In alternate embodiments, search engine 114 and the underlying operatingsystem services may be fully integrated. In other embodiments, somefunctions of search engine 114 and the underlying operating systemservices may be offered by an intermediate “middle” layer of servicesinstead.

For the embodiment, search services 104 are complemented by at least oneproxy server 106, which is coupled to websites 108, clients 102, andsearch services 104 via network 110. Proxy server 106 includes inparticular proxy function 116, and associated query parameter values 117for pre-filling query forms to be provided to their websites 108. Morespecifically, for the earlier described HTML and HTTP orientedembodiment, pre-filled query forms facilitate posting of the associatedquery parameters to their websites 108 via the POST method.

The term “form” as used herein in the specification and in the claims,in general, refers to an electronic form which may be one of any one ofa number of form types, including but are not limited to, HTML FORMs,Windows FORMs, ASP generated FORMs, Java FORMs, Javascript FORMs,directX FORMs, and so forth, as well as form-like documents and/or datastructures/organizations, unless the context of a particular usageinstance specifically narrows the meaning for the particular usageinstance to a particular form type.

As illustrated, indices 115 that effectively index the dynamicallygenerated query answer web pages, and if applicable, associated queryparameter values 117 for pre-filling query forms are generated inaccordance with proxy web page specifications 119 of the dynamicallygenerated query answer web pages.

Except for indices 115 that effectively index the dynamically generatedquery answer web pages, associated query parameter values 117 forpre-filling query forms (if applicable) to facilitate submission ofassociated query parameters of queries of websites 108, and proxy webpage specifications 119, clients 102 (including browsers 112), searchservices 104 (including search engine 114 and indices 115 that indexstatic web pages), proxy server 106 (including proxy function 116),websites 108 (including dynamic web page generators 118 and theirdatabases (not shown)), and network 110 represent a broad range of theserespective elements known in the art or to be designed. In particular,network 110 may include one or more private and/or public networks, suchas the Internet. Their functions and constitutions are known, andaccordingly will not be further described.

Otherwise, the method of the present invention, proxy web pagespecifications 119, indices 115 that effectively index the dynamicallygenerated query answer web pages, and associated query parameter values117 for pre-filling forms (if applicable) to facilitate submission ofassociated query parameters of queries of websites 108, will bedescribed in turn, in the description to follow.

Initialization/Set Up

FIG. 2 illustrates an overview of a method of the present invention, inaccordance with one embodiment. More specifically, FIG. 2 illustratesthe initialization or set up operations for setting up search services104, and if applicable, proxy servers 106, to facilitate clients 102 toaccess deep web information of websites 108.

As illustrated, for each set of search criteria of interest, block 202,the queries including the associated query parameters, and the mannerthey are submitted to websites 108, e.g. whether through the GET methodor the POST method, are identified, block 204. The identified queriesare then reconciled, block 206.

The above operations are best illustrated by a couple of examples. Thefirst example illustrates how queries of various used car websites ofinterest for various used car searches of an HTML/HTTP based embodimentare identified and reconciled, and the second example illustrates howqueries of various airline websites of interest for various used flightschedule/availability searches of an HTML/HTTP based embodiment areidentified and reconciled.

Used Car Sites

Suppose two used car deep web sites 108 are of interest. These sites areAutomall.com and JoesCars.com, and for ease of understanding, theirdynamically generated web pages are assumed to be HTML based. Moreover,communication with Automall.com and JoesCars.com are assumed to be HTTPbased.

Further, the queries of their query answer web pages have similar queryparameters. More specifically, Automall.com's query parameters are carmake, car model, and user's zip code. JoesCars.com's query parametersare car make, car model, user's zip code and car year or years.

The first step in identifying and reconciling the queries of these sitesis to find a minimum number of query parameters for this group ofqueries. In this example case, JoesCars.com has one more queryparameter, which is the year or years. Since Automall.com does not havethis parameter, it can be eliminated. If JoesCars.com requires the yearor years, the largest range of years may be selected. For example, ifJoesCars.com provides support for years between 1982 and 2002, then theyear range for its query will always be 1982-2002.

Choosing a default constant value for the car year leaves three queryparameters for both sites to be specified. This, however, may be reducedfurther. If it is assumed that every model name is unique, then the usermay be able to make a query with just two pieces of information. Theseare the car model and user's zip code. If the car make is required byboth sites, this will be provided in the query. The user will not,however, be required to enter it in the search engine query.

In various embodiments, during this stage of query identification andreconciliation, the user's options for query parameters may also beexpanded. For example, if a list exists which associates a zip code to acity and state, the process may make plans to allow for a user tosubstitute a city or state in the search engine query. The query that issent to each deep web site 108 will contain the zip code even though theuser entered a city or state. If the list of associated cities andstates for zip codes is used, the minimum number of query parameters forused car sites is still two. However, the parameters are now car modeland user location.

Once the minimum number of query parameters is found, the next step isto identify how the query of each deep web site 108 in the group isinvoked. This involves determining the query syntax and method of theHTTP request and the number and type of parameters required. Most ofthis information can be obtained by examining the HTML within the FORMstatement of the dynamically generated query answer page. Any additionalinformation that is required may be found by executing some samplequeries.

Suppose that in the example given, it is found that Automall.com uses aGET HTTP request method and JoesCars.com uses a POST request method.

Furthermore, Automall.com's URL syntax ishttp://www.automall.com/used.html?make=chevrolet&model=corvette&zip=22043and JoesCars.com's POST parameters are:

Make, Model, Year and Zip

The request method and query syntax for the deep web sites 108 are thenknown. What remains to be understood is how the GET and POST parametersneed to be added. For example, if the make parameter for Automall.com isentered through a text field, then it may be insensitive to the case ofa make name. However, if the make parameter is obtained from a pulldown,there is only one way of representing each make name.

Assume that Automall.com and JoesCars.com list car make and model inuser pulldowns which translate to HTML SELECT statements. All of thepossible combinations of make and model can then be obtained from theHTML. Assume that the Year input for JoesCars.com is a TEXT field and“1982-2002” is an acceptable range syntax. Finally, assume that bothsites accept a five-digit zip code through a TEXT field. The method forinvoking the deep web sites is then well understood.

After defining the minimum set of query parameters and understanding themethods of invocation of each deep web site 108, the next step is todevelop a list of site queries and parameter values.

For example, an excerpt from Automall.com's list is illustrated in FIG.3 a. The list contains the query URL followed by a set of queryparameter values that a user might enter as a set of search criteria.

Note that the query URL contains the car make, but the set of queryvalues does not. This is because the minimum set of query parameters waschosen. Note also that some query values contain the city name insteadof the zip code, but the zip code was entered in the URL query. This isan example of the expansion of the possible query parameter values byusing an additional file that associates zip codes and city names.

An excerpt from JoesCars.com list of site queries 302 and parametervalues 304 is also shown in FIG. 3 a.

The first noticeable difference between Automall.com's list andJoesCars.com's list is the URL destination. Instead of JoesCars.com'sURL, the destination is a proxy server's URL (oursite.com). This isbecause of the HTTP request method used on the JoesCars.com site. Asearch engine cannot execute a POST request from one of its links. As aresult, the link given to the search engine must be a GET request to athird party proxy site that can execute the post and return theJoesCars.com results page to the user.

Another difference is the additional Year parameter. As mentioned above,all queries are given the default range of “1982-2002” for thisadditional parameter. Also, since each site might span a different rangeof years the list of possible query parameter values might be different.For example, JoesCars.com might show Ford Granada while this model mightbe too old for Automall.com. Remember that in this example the carmodels were found by searching the SELECT statements of the HTML.

Note that while these lists might contain all possible inputs for theminimum set of query parameters, not all items in the list may besubmitted to the search engine. Instead, each item on the list might becompared to previous search engine queries for relevance. If such aquery is never entered, it may be excluded from the information given tothe search engine.

Airline Reservation Sites

Suppose that two deep web airline reservations sites 108 are of interestfor various flight schedule and availability searches. One is a generaltravel site that lists the reservations of many different airlines. Thissite is called GeneralTravel.com. The other is a site for a specificairline. This site is called FastJet.com.

Again, for ease of understanding, the dynamically generated web pages ofGeneralTravel.com and FastJet.com are assumed to be HTML based, andcommunications with the two sites are HTTP based.

As with the used cars, the first step in identifying and reconciling thequeries of the websites is to identify the minimum set of queryparameters. For airline reservations the minimum set of parameters isfour. These are the “departure date”, “from location”, “return date” and“to location”. These parameters may also be used for hotel and carrental reservations. However, the “from location” is not needed forthese types of reservations.

Suppose GeneralTravel.com also requires the number of people traveling.Since this is not part of the minimum set of parameters, the default canbe used. In this case the default is one person.

The next step is to understand the method of query invocation on eachsite. Suppose that both GeneralTravel.com and FastJet.com use the GETHTTP request method. Also suppose that the URL syntax for each of thesesites is as follows.

http://www.peneraltravel.com/res.html?persons=1&from=bwi&to=sfo&leave=0504&return=0506http://www.fastiet.com/res.html?fdest=bwi&tdest=sfo&start=0504&end=0506

The third step in the process is then to create the lists of sitequeries and parameter values. In the case of used car sites, the carmodels were limited by the values placed in the HTML select statements.Even the number of user zip codes was a finite set. However, the numberof possible departure dates and return dates is infinite. As a result,the list of possible parameter values must be limited by some othermeans.

Suppose that most reservations are booked no more than one year inadvance. Also, suppose that most travel takes place within one month.The possible parameter values are now considerably limited. First ofall, the departure dates are then the next 365 days. For each departuredate, there will be 30 return dates covering the following 30 days.Thus, there will be a total of 365 times 30, or 10,950 possiblecombinations of dates.

To complete the list of values, a list of all possible destinations mustbe available. Every possible combination of to and from destination canthen be generated. Furthermore, the destinations can be specified ascities or airport codes. The complete list of parameter values is then10,950 possible combinations of dates for each possible combination ofto and from destinations.

An excerpt from GeneralTravel.com's list of site queries 312 andparameter values 314 is also shown in FIG. 3 b.

Note that the URL contains information not in the query. In this case,this is the “number of persons traveling.” Also, note that query maycontain the city or the airport, but the URL only contains the airportcode. This implies that a list correlating airport codes and city nameswas used to generate the list of parameter values.

Referring back to FIG. 2, once the queries of the websites 108 ofinterest have been identified and reconciled, corresponding proxy webpage specifications 119 for the dynamically generated query answer webpages are created, block 208, and submitted to search services 104 forprocessing to create indices 115 to index the dynamically generatedquery answer web pages, block 210.

More specifically, as earlier described, corresponding proxy web pagespecifications 119 are submitted to index generator 113, whichgenerates, in response, the earlier described indices 115.

As described earlier, index generator 113 may be any one of a number ofsuch elements known in the art or to be designed. Its functions andconstitution are known, and except for its usage, the manner it isconstituted is not essential to the present invention.

FIGS. 4 a-4 b illustrate one each, an example proxy web pagespecification 119 a-119 b for the queries of Automall.com andGeneraltravel.com. In alternate embodiments, the information may beprovided to search services 104 via other formats and/or approaches.

FIGS. 5 a-5 b illustrate the corresponding example indices that may begenerated to effectively index the dynamic query answer pages for theearlier described search examples, i.e. used car search, and flightschedule and availability search. As illustrated, example indices 500 amay comprise indices 502 a-502 b that advantageously index thedynamically generated query answer pages for the example used carsearch, and example indices 500 b may comprise indices 502 c-502 d thatadvantageously index the dynamically generated query answer pages forthe example flight schedule/availability search, and thereby overcomingthe prior art deficiency of not having any index that indexesdynamically generated web pages.

Referring again to FIG. 2, and recalling from earlier descriptions, forqueries of the HTML and HTTP embodiment employing the FORM and the POSTmethod to post their associated query parameters to their websites 108,associated query parameter values 117 for pre-filling query forms arecreated and pre-stored in proxy server 106 to facilitate the formfilling and parameter value posting. Accordingly, upon creation ofappropriate proxy web page specifications 119 and their provisions toindex generator 113 to generate indices 115, proxy web pagespecifications 119 are employed to create the associated query parametervalues 117 and pre-filling forms and pre-store them with proxy server106 for use during operation.

Associating query parameter values 117 for pre-filling query forms maybe stored on proxy servers 106 using any one of a number of datastructures and/or organizations known in the art or to be designed,including but are not limited to hash tables, flat files, relationaldatabases, and so forth.

FIGS. 6 a and 6 c illustrate one each, a source view of an examplepre-filled form 117 a for the above described used car search example,and a source view of an example pre-filled form 117 b for the abovedescribed flight reservation example, to facilitate posting ofassociated query parameters of queries corresponding to two dynamicallygenerated query answer pages.

As illustrated, pre-filled form 117 a has the values “Ford” and “Focus”pre-selected for the Make and Model parameters. Further, pre-filled form117 a has the zip code value “22043” pre-set for the zip code parameter.

Similarly, pre-filled form 117 b has the parameter values for thedeparting city, destination city, number of passengers, whether it is around trip reservation, departure date, return date, and so forthpre-filled.

For ease of understanding, some of the less illustrative source codes(in the “blank” areas) are omitted.

FIGS. 6 b and 6 d illustrate the end user view of the correspondingexample pre-filled forms 117 a and 117 b for the above described usedcar and flight reservation search examples, when a user selects theindex that effectively indexes the corresponding dynamically generatedquery answer web page.

Those skilled in the art will appreciate that the examples have beenpurposely kept simple for ease of understanding. Nevertheless, theyconvey the substances of the present invention, even though in practice,typically more query parameters and query values are involved.

Operation

Thus, during operation, on receipt of a set of search criteria from aclient 102, search service 104 is able to return a search answer pagecomprising indices indexing query answer web pages that are notpersistent, and are to be dynamically generated.

As described earlier, example search result web pages for the earlierdescribed examples, i.e. used car search and flight schedule andavailability search may comprise indices 502 a-502 b and 502 c-502 dthat advantageously index dynamically generated web pages, therebyovercoming the prior art deficiency of not having any index that indexesdynamically generated web pages.

Further, on selection, the search criteria will be carried forward, andincorporated into the dynamically generated query answer web page,thereby also overcoming the prior art disadvantage of requiring the userto re-enter the search criteria. As described earlier, this iseffectuated through the use of proxy server 106 and the complementaryassociated query parameter values 117 for pre-filling query formspre-generated and pre-stored on proxy server 106.

In alternate embodiments, in lieu of retrieving the relevant forms inreal time from the applicable websites 108, the present invention may bepracticed with the forms pre-fetched, pre-filled and cached.

For the illustrated embodiment, preservation and transfer of the searchcriteria or query parameters are accomplished without resorting to theuse of cookies (and client approval of such use). However, in alternateembodiments, some or all of the query parameters may be preserved and/ortransferred involving the use of cookies (and client approval of suchuse).

Moreover, the present invention may be practiced with the preservationand transfer of other state information, in addition to the searchcriteria. Furthermore, the present invention may also be practiced inconjunction with the employment of one or more security techniques.

Example Computer System

FIG. 7 illustrates an example computing system suitable for use aseither a client or to host the proxy function, the search engine or aweb server to practice the present invention. As a client system,computing system 700 may be a desktop computer system, a laptop computersystem, a tablet computing device, a palm sized computing device, awireless mobile phone, a set-top box, an Internet appliance and thelike. As a server, computing system 700 may be a single or a cluster ofcomputer systems.

As shown, exemplary computing system 700 includes one or more processors702 and system memory 704. Additionally, system 700 may include massstorage devices 706 (such as diskette, hard drive, CDROM and so forth),input/output devices 708 (such as keyboard, cursor control and so forth)and communication interfaces 710 (such as network interface cards,modems and so forth). The elements are coupled to each other via systembus 712, which represents one or more buses. In the case of multiplebuses, the buses are bridged by one or more bus bridges (not shown).

Each of these elements performs its conventional functions known in theart. In particular, system memory 704 and mass storage 706 are employedto store a working copy and a permanent copy of the programminginstructions and/or data, implementing the teachings of the presentinvention (e.g. the enhanced indices of the search engine, thecomplementary pre-filled forms, and so forth). The programminginstructions may be instructions of any one of a number of known or tobe designed languages, including but are not limited to C, C++, Perl,Java, Javascript, XML, HTML and so forth. All or a portion of thepermanent copy of the programming instructions/data may be loaded intomass storage 706 in the factory, in the field (via a distributionmedium) or through communication interface 710 (from a distributionserver (not shown)).

The constitution of these elements 702-712 are known, and accordinglywill not be further described.

Additional Embodiments

In additional embodiments, illustrated by FIGS. 10-15, the searchservices and the proxy server may be effectively combined, by having thesearch services automatically select the indices that index thedynamically generated web pages, and cause the query result web pages tobe embedded as part of the search result page. As can be seen from FIGS.10, 12, and 14, embedded query result pages may be included as answersof the search results page. Such answers may include one or more inputfields. In some embodiments, one or more of these fields may have valuesentered corresponding to one or all of the search criteria of the usersearch request. Other fields may be left empty or contain defaultvalues, and may thus enable a user to enter new values or alter defaultor previously entered values.

In one embodiment, the query results may be embedded into the searchresult page as other “hits”/answers, and placed as a group above, below,or among the other “normal” hits/answers. Example search result pages1002 a, 1202 a, and 1402 a of FIGS. 10 a, 12 a, and 14 a, respectively,illustrate such an embodiment. In another embodiment, the query resultsmay be embedded into the search result page as hits/answers, and may beinterspersed among the other “normal” hits/answers. Example searchresult pages 1002 b, 1202 b, and 1402 b of FIGS. 10 b, 12 b, and 14 b,respectively, illustrate such an embodiment. In a third embodiment, thequery results may be embedded into the search result page a singlehits/answer, the single hit/answer providing an embedded query to serveas a proxy for a plurality of query result pages. Example search resultpages 1002 c, 1202 c, and 1402 c of FIGS. 10 c, 12 c, and 14 c,respectively, illustrate such an embodiment. In yet another embodiment,the query results may be embedded into the search result page ashits/answers, and may be placed along the side of the other “normal”hits/answers. Example search result pages 1002 d. 1202 d, and 1402 d ofFIGS. 10 d, 12 d, and 14 d, respectively, illustrate such as embodiment.

As is shown by FIGS. 10 a-10 d, embodiments of the present invention maymake available deep web information associated with vehicular rentalservices to users by embedding query result pages of informationlocations associated with vehicular rentals as hits/answers of a searchresults page, the search results page having a plurality of answers.Each hit/answer embedding a query result page may include an input fieldfor a rental location, fields for a pick-up date and time, field for areturn date and time, and a field for a vehicle type. As shown, thetimes and vehicle type may be selected from a pre-determined choice listof options using a select box. In alternate embodiments, however any ofthe fields may be implemented as input text fields, select box lists,radio buttons, or any other form elements known in the art. Also, asillustrated, each hit/answer embedding a query result page may include aform element or hyperlink (in FIGS. 10 a-10 d, shown as the “Check” box)effectively indexing to the information location having the query resultpage that is embedded by the search results page. When actuated by auser, the values entered into the input elements of the form may betransmitted to the information location, and the information locationmay dynamically generate the query results page, including thetransmitted values in its fields. Also, as is discussed above, thehits/answers embedding query result pages may be included as a groupabove the other normal hits/answers (FIG. 10 a), may be interspersedamong the other normal hit/answers (FIG. 10 b), may be presented as asingle hit/answer serving as a proxy for a plurality of query resultpages (FIG. 10 c), or may be presented along side of the other normalhits/answers (FIG. 10 d). Also, FIG. 11 illustrates an exemplaryfragment of source code of a search result page, the page associatedwith a car rental service query.

As illustrated by FIGS. 12 a-12 d, embodiments of the present inventionmay make available deep web information associated with trainreservation services to users by embedding query result pages ofinformation locations associated with train reservations as hits/answersof a search results page, the search results page having a plurality ofanswers. Each hit/answer embedding a query result page may include aninput text field for a departure location, an input text field for anarrival location, and an input text field for a travel date. Inalternate embodiments, any of the fields may be implemented as inputtext fields, select box lists, radio buttons, or any other form elementsknown in the art. Also, as illustrated, each hit/answer embedding aquery result page may include a form element or hyperlink (in FIGS. 12a-12 d, shown as the “Check” box) effectively indexing to theinformation location having the query result page that is embedded bythe search results page. When actuated by a user, the values enteredinto the input elements of the form may be transmitted to theinformation location, and the information location may dynamicallygenerate the query results page, including the transmitted values in itsfields. Also, as is discussed above, the hits/answers embedding queryresult pages may be included as a group above the other normalhits/answers (FIG. 12 a), may be interspersed among the other normalhit/answers (FIG. 12 b), may be presented as a single hit/answer servingas a proxy for a plurality of query result pages (FIG. 12 c), or may bepresented along side of the other normal hits/answers (FIG. 12 d). Also,FIG. 13 illustrates an exemplary fragment of source code of a searchresult page, the page associated with a train reservation service query.

As illustrated by FIGS. 14 a-14 d, embodiments of the present inventionmay make available deep web information associated with flight statusinformation services to users by embedding query result pages ofinformation locations associated with flight status information ashits/answers of a search results page, the search results page having aplurality of answers. Each hit/answer embedding a query result page mayinclude an input text field for a flight number and an input text fieldfor a flight date. In alternate embodiments, any of the fields may beimplemented as input text fields, select box lists, radio buttons, orany other form elements known in the art. Also, as illustrated, eachhit/answer embedding a query result page may include a form element orhyperlink (in FIGS. 14 a-14 d, shown as the “Check” box) effectivelyindexing to the information location having the query result page thatis embedded by the search results page. When actuated by a user, thevalues entered into the input elements of the form may be transmitted tothe information location, and the information location may dynamicallygenerate the query results page, including the transmitted values in itsfields. Also, as is discussed above, the hits/answers embedding queryresult pages may be included as a group above the other normalhits/answers (FIG. 14 a), may be interspersed among the other normalhit/answers (FIG. 14 b), may be presented as a single hit/answer servingas a proxy for a plurality of query result pages (FIG. 14 c), or may bepresented along side of the other normal hits/answers (FIG. 14 d). Also,FIG. 15 illustrates an exemplary fragment of source code of a searchresult page, the page associated with a flight status service query.

In one embodiment, rather than embedding the exact query result web pagein the search answer page, an approximation of the query result page maybe embedded. For example, the exact query result page may includeseparate fields for a month and a date, while the embedded entry mayinclude one combination field for the month and date. In such anembodiment, a transformation program may be applied to the search answerpage's embedded submission, splitting, for example, the submittedmonth-year combination. The split submission may then be furthersubmitted to the query result web site. Such a transformation programmay be a JavaScript program or a program of any programming languageknown in the art. Such a transformation program may be a proxy betweenthe search engine of the search results page and the target web site ofthe query result web page. In such a case, where the transformationprogram is a proxy, the HTTP request that a browser of the userdisplaying the search answer page generates may be redirected throughthe proxy, the proxy converting the combined form value, such as thecombined month-date, of the HTTP request to the desired format, such asa split month and date, and sending the converted HTTP request to thetarget site. Also, the same (or different) proxy may receive theresulting web page from the target site and forward it back to thebrowser.

Also, embodiments of the present invention may make available deep webinformation associated with one or more of limousine reservationservices, taxi reservations services, bus reservation/schedule services,both local and long distance, boat reservation/schedule services, shipreservation/schedule services, car rental location services, trainstatus information services, ferry reservation/schedule services, andcruise liner reservation/schedule services to users by embedding queryresult pages of information locations associated with the service(s) ashits/answers of a search results page.

Alternate Embodiments

While the present invention has been described in terms of the foregoingembodiments, those skilled in the art will recognize that the inventionis not limited to the embodiments described. The present invention canbe practiced with modification and alteration within the spirit andscope of the appended claims.

In particular, while for ease of understanding, the search services andthe proxy server have been described as two separate entities, inalternate embodiments, the two may be combined, or one or more functionsof one or more of search services and proxy server may be distributedamong additional implementation entities.

Further, in alternate embodiments, the present invention may also besuitably implemented, at least in part, in firmware or hardware, or somesuitable combination of at least two of the three. Such firmware orhardware embodiments may include programmable logic devices, such asprogrammable array logic (PALs) and programmable logic arrays (PLAs),field programmable gate arrays (FPGAs), application specific integratedcircuits (ASICs), large scale integrated circuits (LSIs), very largescale integrated circuits (VLSI) or the like—to form the various typesof modules, circuitry, controllers, routines and systems described andclaimed herein.

CONCLUSION AND EPILOGUE

Thus, it can be seen from the above descriptions, a novel method,including associated apparatuses, for accessing deep web informationassociated with transportation services has been described. Thedescription is to be regarded as illustrative instead of restrictive onthe present invention, which scope is set forth in the claims to follow.

What is claimed is:
 1. A method, comprising: upon receipt of a set ofsearch criteria from a client, generating a search answer page includinga first answer including a uniform resource locator (URL) and a non-URLportion, the URL containing a first portion identifying a web page and asecond portion associated with the search criteria and useable by theweb page to dynamically generate content relating to the searchcriteria; and returning the search answer page to the client.
 2. Themethod of claim 1 wherein the search answer page has one or more inputfields included therein when returned to the client.
 3. The method ofclaim 1 wherein the second portion of the URL is useable by the web pageto pre-fill a form.
 4. The method of claim 2, wherein one or more of theinput fields have values entered corresponding to the set of searchcriteria.
 5. The method of claim 4, wherein others of the input fieldsare left empty.
 6. The method of claim 4, wherein others of the inputfields contain default values.
 7. The method of claim 4, wherein theinput fields are included in the search answer page above other searchresults in the search answer page.
 8. The method of claim 4, wherein theinput fields are included in the search answer page interspersed amongother search results in the search answer page.
 9. The method of claim2, wherein the input fields are included in the search answer page as asingle search result, the single search result providing an embeddedquery to serve as a proxy for a plurality of query result pages.
 10. Themethod of claim 2, wherein the input fields are included in the searchanswer page placed alongside other search results.
 11. The method ofclaim 2, wherein at least some of the input fields represent informationassociated with one or more of flight status information services,vehicular rental services, limousine reservation services, taxireservations services, bus reservation/schedule services, trainreservation/schedule services, boat reservation/schedule services, shipreservation/schedule services, car rental location services, trainstatus information services, ferry reservation/schedule services, andcruise liner reservation/schedule services to users by embedding queryresult pages of information locations associated with the service(s) asanswers in the search answer page.
 12. The method of claim 11, wherein,at the direction of a user, values entered into the input fields aretransmitted to the information locations, query results are dynamicallygenerated at the information locations, and said query results arereturned to the client.
 13. The method of claim 2, wherein the searchanswer page embeds a query result page including: an input element for alocation and an input element for a travel date.
 14. The method of claim13, wherein, at the direction of a user, values entered into one or moreof the input elements are transmitted to information locationsassociated with at least one of flight status information services,vehicular rental services, limousine reservation services, taxireservations services, bus reservation/schedule services, trainreservation/schedule services, boat reservation/schedule services, shipreservation/schedule services, car rental location services, trainstatus information services, ferry reservation/schedule services, andcruise liner reservation/schedule services, and the informationlocations dynamically generate further query results pages, includingthe transmitted values in its fields.
 15. The method of claim 2,wherein, the search answer page is generated in response to atransformation program submitting values for inclusion in input fieldsin a form, and the transformation program is a proxy between a searchengine that produces the search answer page and a target web site of aresult included on the search answer page, and an HTTP request that abrowser running on the client displaying the search answer web pagegenerates is redirected through the proxy, the proxy converting a formvalue of the HTTP request to a desired format and sending the HTTPrequest to the target web site.
 16. A non-transitory computer-readablemedium having stored thereon processor-executable instructions, whichinstructions, when executed by a processor, cause the processor toperform steps, comprising: upon receipt of a set of search criteria froma client, generating a search answer page including a first answerincluding a uniform resource locator (URL) and a non-URL portion, theURL containing a first portion identifying a web page and a secondportion associated with the search criteria and useable by the web pageto dynamically generate content relating to the search criteria; andreturning the search answer page to the client.
 17. The non-transitorycomputer-readable medium of claim 16, wherein the search answer page hasone or more input fields included therein when returned to the client.18. The non-transitory computer-readable medium of claim 16 wherein thesecond portion of the URL is useable by the web page to pre-fill a form.19. The non-transitory computer-readable medium of claim 17, wherein oneor more of the input fields have values entered corresponding to the setof search criteria.
 20. The non-transitory computer-readable medium ofclaim 19, wherein others of the input fields are left empty.
 21. Thenon-transitory computer-readable medium of claim 19, wherein others ofthe input fields contain default values.
 22. The non-transitorycomputer-readable medium of claim 19, wherein the input fields areincluded in the search answer page above other search results in thesearch answer page.
 23. The non-transitory computer-readable medium ofclaim 19, wherein the input fields are included in the search answerpage interspersed among other search results in the search answer page.24. The non-transitory computer-readable medium of claim 17, wherein theinput fields are included in the search answer page as a single searchresult, the single search result providing an embedded query to serve asa proxy for a plurality of query result pages.
 25. The non-transitorycomputer-readable medium of claim 17, wherein the input fields areincluded in the search answer page placed alongside other searchresults.
 26. The method of claim 17, wherein at least some of the inputfields represent information associated with one or more of flightstatus information services, vehicular rental services, limousinereservation services, taxi reservations services, busreservation/schedule services, train reservation/schedule services, boatreservation/schedule services, ship reservation/schedule services, carrental location services, train status information services, ferryreservation/schedule services, and cruise liner reservation/scheduleservices to users by embedding query result pages of informationlocations associated with the service(s) as answers in the search answerpage.
 27. The non-transitory computer-readable medium of claim 26,wherein, at the direction of a user, values entered into the inputfields are transmitted to the information locations, query results aredynamically generated at the information locations, and said queryresults are returned to the client.
 28. The non-transitorycomputer-readable medium of claim 17, wherein the search answer pageembeds a query result page including: an input element for a location,and an input element for a travel date.
 29. The non-transitorycomputer-readable medium of claim 28, wherein at the direction of auser, values entered into one or more of the input elements aretransmitted to information locations associated with at least one offlight status information services, vehicular rental services, limousinereservation services, taxi reservations services, busreservation/schedule services, train reservation/schedule services, boatreservation/schedule services, ship reservation/schedule services, carrental location services, train status information services, ferryreservation/schedule services, and cruise liner reservation/scheduleservices, and the information locations dynamically generate furtherquery results pages, including the transmitted values in its fields. 30.The non-transitory computer-readable medium of claim 17, wherein, thesearch answer page is generated in response to a transformation programsubmitting values for inclusion in input fields in a form, and thetransformation program is a proxy between a search engine that producesthe search answer page and a target web site of a result included on thesearch answer page, and an HTTP request that a browser running on theclient displaying the search answer web page generates is redirectedthrough the proxy, the proxy converting a form value of the HTTP requestto a desired format and sending the HTTP request to the target web site.